Exploratory HJB Equations and Their Convergence

We study the exploratory Hamilton–Jacobi–Bellman (HJB) equation arising from entropy-regularized control problem, which was formulated by Wang, Zariphopoulou, and Zhou (J. Mach. Learn. Res., 21 (2020), 198) in context of reinforcement learning continuous time space. establish well-posedness regularity viscosity solution to equation, as well convergence problem classical stochastic when level exploration decays zero. then apply general results obtained temperature introduced Gao, Xu, (SIAM J. Control Optim., 60 (2022), pp. 1250–1268) design an endogenous schedule for simulated annealing nonconvex optimization. derive explicit rate this diminishes zero, find that stationary distribution optimally controlled process exists, is however neither a Dirac mass on global optimum nor Gibbs measure.